Evaluating Perceptual Judgements on 3D Printed Bar Charts

A case for new data visualizations

Tyler Wiederich and Susan VanderPlas

5/25/23

Today’s Topics

  • Background on 3D graphics
  • A study with 3D printed graphs
  • Experiential learning

Introduction

Background on 3D graphics

History is filled with data visualizations, but some of the earliest references to 3D visualizations were in 1874 by James Clerk Maxwell.

Since then, studies have shown that 3D graphics are less effective at conveying information when a 2D equivalent is available. As of now, most of those studies focus on 2D projections of 3D graphics.

Background on 3D graphics

When to use 3D graphics

  • Relationship between multiple variables

  • Display data over 3D objects

When to NOT use 3D graphics

  • Does not add any additional information

  • Lack of interactivity

A Study on 3D Printed Graphics

Overview

Research questions:

  1. How do 3D printed charts compare to 2D and 3D digital charts?
  2. Can we replicate the findings from Cleveland and McGill?

Replicating Cleveland and McGill

Our study draws influence from Cleveland and McGill’s 1984 paper, specifically from their position-length experiment with graph types 1 and 3.

  • Types 1-3 measure position, and types 4-5 measure length

Graph types for position-length experiment from Cleveland and McGill

In their study, participants were asked which of the marked bars were smaller and by approximately how much.

Replicating Cleveland and McGill

Values involved in comparison judgments are

\[V_i=10\cdot10^{(i-1)/12}\quad i=1,\dots, 10\]

To closely replicate values, we made two assumptions about the comparisons from Cleveland and McGill

  1. Match the exact ratios that were used

  2. No value was used more than twice

Study Design

Treatment Design

  1. Seven unique ratios (17.8, 26.1, 38.3, 46.4, 56.2, 68.1, 82.5)
  2. Three graph types (2D digital, 3D digital, 3D printed)
  3. Two comparisons: Adjacent (Type 1) or separated (Type 3)

There are 42 treatment combinations, too many for a busy subject!

  • Each subject receives five of the seven unique ratios
  • Within each ratio, all three graph types are used
  • Each graph is then randomly assigned an adjacent or separated comparison
  • Now 15 treatment combinations per subject

Design of each kit

Study Design

Each subject receives a bag of 3D printed kits with instructions that direct users to a Shiny app

Shiny App

Shiny App

Shiny App

Shiny App

Shiny App

Shiny App

Model

\[y_{ijklm}=\mu+S_i+R_j+G(R)_{(k)j}+T_l+\epsilon_{ijklm}\]

where

  • \(y_{ijklm}=\log_2(|\text{Judged Percent} - \text{True Percent}|+1/8)\)

  • \(S_i\sim N(0,\sigma^2_S)\) is the effect of the \(i^{th}\) subject

  • \(R_j\) is the effect of the \(j^{th}\) ratio

  • \(G(R)_{(k)j}\) is the effect of the \(k^{th}\) graph type nested in the \(j^{th}\) ratio

  • \(T_l\) is the effect of the \(l^{th}\) comparison type

  • \(\epsilon_{ijklm}\sim N(0,\sigma^2_\epsilon)\) is the random error

Results

  • Few responses incorrectly identified the smaller bar and were removed from the data
Term SS MS Num DF Den DF F-value P-value
ratio 0.403 0.403 1 512.170 0.233 0.629
type 3.002 3.002 1 521.459 1.740 0.188
ratio:plot 0.056 0.028 2 508.348 0.016 0.984
  • Plot within ratio was not not statistically significant, nor was anything else
  • Cleveland and McGill showed an increase in error from Type 1 to Type 3

Bootstrap Confidence Intervals

In Cleveland and McGill, confidence intervals were calculated using bootstrap sampling of subjects using the means of the midmeans. The same process is used here, although each subject did not receive each treatment combination.

A Contemporary Approach with GLMMs

The responses from the previous analysis used the log errors, but Generalized Linear Mixed Models are a modern option to analyze the data. Adjusting for the ratios as covariates, a beta regression model with unequal slopes is fit below:

\[\eta_{ijkl}=S_i+\beta_j R_{jk}+G_k+T_l\]

  • \(\eta_{ijkl}\) is the participant response

  • \(S_i\sim N(0,\sigma^2_S)\) is the effect of the \(i^{th}\) subject

  • \(\beta_j\) is the effect of the \(j^{th}\) true ratio

  • \(R_{jk}\) is the \(j^{th}\) ratio for the \(k^{th}\) graph type

  • \(T_l\) is the effect of the \(l^{th}\) comparison type

  • \(y_i|S_i\sim Beta(\mu_{ijkl}, \phi)\)

  • Link function: \(\eta_{ijkl}=\log[\mu_{ijkl}/(1-\mu_{ijkl})]\)

GLMM Results

plot _plot Estimate Standard Error DF t-Value Adj P Odds Ratio Adj Lower Odds Ratio Adj Upper Odds Ratio
2dDigital 3dDigital 0.21 0.06263 491 3.35 0.0025 1.234 1.065 1.429
2dDigital 3dPrint 0.07093 0.06164 491 1.15 0.4834 1.074 0.929 1.241
3dDigital 3dPrint -0.1391 0.06185 491 -2.25 0.0642 0.87 0.752 1.006

Beta regression does not have easily interpretable results.

\(\frac{\mu_{ij1l}/(1-\mu_{ij1l})]}{\mu_{ij3l}/(1-\mu_{ij3l})]}\)

Future work

Changes to Study Design

  • Include static 3D charts (similar to Microsoft Excel)
  • Explore other types of visualizations, such as 3D printed heat maps

Integration with Introductory Statistics Course

No results were significant, but what if we could incorporate the study at a larger scale?

Experiential learning with STAT 218 at University of Nebraska-Lincoln.

  1. Participate in the experiment
  2. Reflect on the experimental design and how randomization occurred
  3. Read a paper on the experiment from the perspective of the researchers and reflect

Questions?